A Parallel N -Body Data Mining Framework

The N-body or multi-tree approach for accelerating data mining methods has spurred some of the fastest known solutions for a significant class of fundamental methods. We present a standard mathematical model and associated programming model that allows these problems to be scaled further via parallelization, without significant extra programmer effort. With the framework, we derive a strategy for multi-core and cluster parallelism and create an implementation, Tree-based High-Order Reduce or THOR. We demonstrate three realistic parallel applications: kernel density estimation on the Sloan Digital Sky Survey, two-point correlation of galactic halo positions, and nearest neighbor search on high-dimensional speech data. Many powerful nonparametric methods that take full advantage of massive data require analyzing all pairs of distances ; however, such analysis is time consuming. For a significant subclass of these problems, which we call generalized N-body problems (GNP's), a higher-order divide-and-conquer technique enumerated in [6] can avoid nearly all computation by skipping combinations of regions with little mutual interaction. Higher-order divide and conquer on metric data typically utilizes space-partitioning trees, such as kd-trees [4], ball trees [19], and cover trees [1], to assist in dividing the data into regions. This use of trees is called the dual-tree or the broader multi-tree approach. This approach generalizes long-used algorithms such as kd-tree-based nearest-neighbor search [5], as well as more recent algorithms using space-partitioning trees for N-body physical simulation such as the celebrated Fast Multipole Method (FMM) [8] and for statistical problems like kernel regression [20]. For nearest-neighbor search and kernel regression, the dual-tree approach generalizes these earlier algorithms by considering all query points simultaneously, by organizing them in a second tree. The multi-tree approach generalizes the FMM by casting it recursively, allowing arbitary tree structures, recursion patterns, approximation schemes, and error criteria. This approach is applied to many statistical and physical problems on metric data, such as bounded-error kernel density estimation [6, 7, 15, 14], k-nearest-neighbor classification [17], kernel discriminant analysis [21], nearest neighbor search [6], n-point correlation [6, 18], and more. For each of these problems, no overall faster serial algorithms are known. Other machine learning problems treated with this strategy include dimensional-ity reduction methods [9], nonparametric belief propagation [10], linear algebraic machine learning methods [2], and particle filters [13]. Many current and future data mining techniques remain to be considered in depth with this strategy, not to mention a large array of related fundamental problems in computational geometry, physics, and …